ZO-JADE: Zeroth-order Curvature-Aware Distributed Multi-Agent Convex Optimization

In this work we address the problem of convex optimization in a multi-agent setting where objective is to minimize mean local cost functions whose derivatives are not available (e.g. black-box models). Moreover agents can only communicate with neighbors according connected network topology. Zeroth-order (ZO) has recently gained increasing attention federated learning and scenarios exploiting finite-difference approximations gradient using from $2$ (directional gradient) $2d$ (central difference full evaluations functions, $d$ dimension problem. The contribution extend ZO distributed by estimating curvature via approximations. particular, propose novel algorithm named ZO-JADE, that adding just one extra point, i.e. $2d+1$ total, allows simultaneously estimate diagonal Hessian, which then combined average tracking consensus obtain an approximated Jacobi descent. Guarantees semi-global exponential stability established separation time-scales. Extensive numerical experiments on real-world data confirm efficiency superiority our respect several other zeroth-order methods literature based estimates.